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We consider the problem of transmission of a sequence of real data produced by a Nyquist sampled band- 
limited analog source over a band-limited analog channel, which introduces an additive white Gaussian noise. 
An analog coding scheme is described, which can achieve a mean-squared error distortion proportional to (1 + 
SNR)~ B for a bandwidth expansion factor of B/R, where < R < 1 is the rate of individual component binary 
codes used in the construction and B ^ 1 is an integer. Thus, over a wide range of SNR values, the proposed code 

CI 

performs much better than any single previously known analog coding system. 
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I. Introduction 



We consider the problem of transmission of a sequence of real data produced by a band-limited analog 
source over a band-limited analog channel, which introduces an additive white Gaussian noise. A similar 
J> ■ setting arises in several practical situations. A traditional approach towards solving this problem is through 
■ the application of Shannon's source-channel separation principle. This involves the separate design of 
^ [ optimal (digital) source and channel codes, and using these codes in tandem [MP02]. However by invoking 
00 \ the separation principle, we sacrifice one of the most important advantages of an analog communication 
^ ■ system, namely its ability to perform well under varying noise levels. In literature, this property has been 
q . referred to as "robustness". By contrast, a digital communication system employing separate source and 
\ channel codes typically can operate satisfactorily only within a very narrow region around the designed 
; signal to noise ratio. Another important property of an analog system is the graceful degradation in 
^ ■ performance with an increasing noise level. This property is very valuable for a good broadcast system. 
^ ! However analog systems have several disadvantages too. In practice an analog system is complicated 
^ \ to design and maintain, is susceptible to drift with time, and expensive. In addition, practical systems 
- - ' thus far suffer from the threshold-effect - beyond a certain signal-to-noise ratio, the system performance 
tends to either saturate or improve only marginally with more transmit power. In spite of several such 
shortcomings, analog systems are still rated high in applications such as transmission and recording of 
music, due to the lack of granularity which is the result of imperfect source quantization. 

In order to compare various analog systems, it is customary to measure the end to end distortion in 
terms of the mean squared error. In an effort to explain the threshold phenomena in analog communication 
systems, Ziv [Ziv70], considered a broad class of well behaved modulation signals which include all 
currently employed analog schemes. He showed that for this class with bandwidth expansion factors over 
unity, as the SNR 7 increases, the mean squared distortion cannot fall at a rate faster than 7~ 2 . Shamai, 
Verdu and Zamir [SVZ98] demonstrated a joint source-channel coding scheme that is optimal for a class 
of sources and channels. In [MP02], Mittal and Phamdo presented several hybrid digital-analog joint 
source-channel codes which are shown to be "nearly robust" for the case of broadcast with two receivers. 
For the case of broadcast to two receivers, a distortion pair {D\, D2) is said to be achievable if the 
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first listener achieves D\ and the second listener D2 simultaneously. In a recent work Reznic, Feder and 
Zamir [RFZ05] consider the problem of analog broadcast to two listeners at bandwidth expansion ratios 
of more than 1. They prove a lower bound for the achievable rate distortion pairs using information 
theoretic arguments. Using some necessary conditions, they show that if a system is optimal at a certain 
high enough 7, then its distortion cannot decay at a rate faster than 7 . Thus one of the Mittal-Phamdo 
schemes is seen to be optimal at high signal to noise ratios. 

In addition to the hybrid code constructions due to Shamai et al [SVZ98], Mittal et al [MP02], and 
Reznic et al [RFZ05], there have been attempts to construct purely analog coding schemes for the general 
analog channel with bandwidth expansion factors of larger than unity. Chen and Wornel [CW98] give 
an analog code construction based on chaotic maps for uniformly distributed analog source, along with 
an efficient decoding algorithm using estimation techniques. Vaishampayan and Costa [VC03] construct 
analog codes using linear dynamical systems and provide efficient decoding algorithms. Using curves on 
higher dimensional spheres and topological arguments they show that their codes have much in common 
with the Chen et al chaos map codes. In both the above constructions, the mean square distortion cannot 
decay at a rate better than 7~ 3 / 2 . 

All these prior results seem to indicate that strong analog coding schemes invariably suffer from the 
analog threshold effect, further limiting their practical usefulness. Shannon's rate distortion theory on the 
other hand suggests that mean square distortion may fall at a rate of 7~ N at a bandwidth expansion factor 
of N. It has been widely conjectured that no single analog (or hybrid analog-digital) scheme can achieve 
this rate of fall with SNR. In this paper, we show that this is not necessarily so. Here we construct a family 
of analog codes, using digital component codes. The component codes are used in a "power-splitting" 
superposition scheme with an infinite number of levels. This is similar in principle to the superposition 
scheme introduced by Cover [Cov72] and related to the multilevel codes considered by Calderbank and 
Seshadri in [CS93]. Assuming unbounded complexity is admissible at both the transmitter and receiver, 
we analyze the performance of these codes on an AWGN channel. We then consider some practical 
component codes and show that under most practical situations, the codes constructed herein can give a 
rate of fall of mean-square-distortion well above 7~ 2 over a wide range of SNRs. It must however be 
noted that our results do not contradict the conclusions of either Ziv [Ziv70] or Reznic et al [RFZ05]. 
Instead the class of codes constructed here do not fall under the type of modulation signals considered 
by Ziv. The analog codes described here achieve a fall in distortion proportional to 7~ B , where B is an 
integer such that 1 ^ B ^ [R ■ N\ , where N is the bandwidth expansion factor and < R < 1. Moreover 
though it never achieves any distortion point which is on the lower bound given by the rate-distortion 
function, the achievable MSE distortion can be made to fall almost parallel to the lower bound at high 
enough SNR. 

In the next section, we give the construction of our code, and several practical examples. We also show 
why the codes we construct do not fall into the class of codes considered by Ziv. In the third section, we 
analyze the performance of the proposed system assuming use of capacity achieving component binary 
codes. We also give a distortion upper bound using the union-bounding technique for hard decision 
maximum likelihood decoding for the component binary codes. We give simulation results for analog 
codes constructed using repeat-accumulate codes as the component codes. The analog decoder used in 
this case employs the sum-product algorithm decoder along with a hierarchical successive cancellation 
technique. 

II. Theoretical Bound from Rate-Distortion Theory 

We consider a discrete time analog source which draws values independently and uniformly from the 
interval S = [— V3, V3). The average source power is thus constrained to be unity. We may assume 
that, such a discrete time memoryless source which outputs at a rate of 2Ws samples per second can be 
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constructed from a band-limited continuous time source by means of appropriate sampling. Let us denote 
by s = {Sf}, the sequence of continuous valued random symbols generated by this source. 

We wish to transmit the source output to listeners across a band-limited and power-limited continuous 
time analog channel which adds a white Gaussian noise to the input signal. The channel can also be 
represented by an equivalent discrete time version using the Nyquist sampling theorem. The channel is 
used at the rate of 2Wq per second. Thus if the channel input, output and noise are denoted by yt, ft 
and Vt respectively, then they are related as r\ = yt + Vt at time instant t. Without loss of generality, the 
channel input is assumed to be subject to the input power constraint Ew[y 2 ] = 1. 

By rate-distortion theory, for a memoryless uniform source, the rate distortion function can be obtained 
as a lower-bound on a difference in differential entropy: 

R(D) = h(s) - h(s\s) = -h(s\s) ^ \ log 2 (j^d) (bits/use) = W s log 2 (bits/s) 

Similarly, the capacity of the discrete-time AWGN channel is: 

C(cr 2 ) = \ log 2 (1 + (bits/use) = W c log 2 (1 + 4y) (bits/s) 

The two relations together give the following lower bound on the achievable mean squared-error distortion 
for analog transmission of a uniform source over the discrete memoryless AWGN channel: 

D(cr 2 ) > 



where, N = W^/Wg is called the bandwidth expansion factor. We call 1/cr 2 the signal to noise (power) 
ratio. 

An analog encoder of length n is a block encoder which maps the real source samples grouped k at a 
time to a vector of real values of length n. Thus the encoder 8% is a mapping : R k 1— > R", such that 
the encoded sequence satisfies the input power constraint of the channel, ^£5 [| |£fc(S) 1 1 2 ] ^ 1, where S 
represents a block of k source samples. Similarly, the analog decoder is a mapping : R w 1— > R . The 
received signal is Z = W + V where W = £k(S). A family of analog codes, is a sequence of encoder- 
decoder pairs with increasing block-length k, and a fixed "rate" R = |. Our objective is to construct a 
family of analog codes which achieve sufficiently low mean squared error distortion in the limit of very 
large block lengths, D(a 2 ) = lim^^ i£s[||S — S|| 2 ], where S is the vector of estimates produced by 
the decoder Vfc. Below we give our code construction. 



III. Construction of the Analog Codes 

Let a realization of the source output at time t be denoted as Sf. Since the source has mean zero and 
unit variance uniform pdf, scale the realization to obtain a uniform distribution in the interval [0,1) by 
forming xt = (st + V3) / (2y/3). Now we represent Xt by its terminating binary representation. Let this 
binary representation be given by X; = {Xi,f,X2,f, • • ■ ,X^ f , ■ • • }. It is well known [Res98] that, Xi /t arc 
Bernoulli random variables with p = 1/2. For a fixed integer parameter B, form for each i E {1, 2, ■ ■ ■ } 
sequences of the form 

U; = { • • • , Xt-l,iB'Xt,(i-l)B+l>Xt,(i-l)B+2/ ' ' ' ' ^t,iB>^t+l,{i-l)B+V^t+l,(i-l)B+2> ' ' ' / } 



Now using component binary codes, encode the sequences Uj for each i separately. Denote by W 2 - 
the resulting sequence of Bernoulli random variables. Each such i is referred to as the i th encoded (bit) 
level. By a suitable choice of the component code, it is always possible to ensure that this sequence is 
composed of Bernoulli random variables that are distributed fairly, with p = 1/2. The encoding of the 
reordered source sample bit planes is shown in Figure 1. 
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Figure 1 The source bit planes are reordered and encoded using component binary codes C{ at each level i. 
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Now define a mapping f w : {0, 1}* — > R for a binary sequence b 6 {0, 1}* as, 

00 

/^^Qa.-ij.fflj (i) 

where, = • (1 — w) 1 ^ for some w G [0, 1). This mapping will be often referred to as the analog 
encoding map. 

The encoded sequence of bits W; are now read off vertically as shown in Figure 2 and the analog 
mapping function f w (.) is applied with some fixed weight, w. The real valued vector y so obtained is 
the analog code corresponding to the source vector s. The analog encoding map satisfies the unity input 
power constraint for the analog channel and the resulting vector is transmitted over the channel. 

As an example, let us take the simplest case, when B = 1. Then, the above code construction would 
be equivalent to encoding each one of the i th bit planes at significant positions i separately, followed by 
the mapping f w {-) which is invoked at each time instant. The analog encoder block diagram is depicted 
in Figure 3. 

IV. The main results 

The following lemma brings out the self similar, scaled nature of the pdf of the real random variables, 
yy produced by the code construction as given in the previous section. 

Lemma IV.l Let W = {W\, W2, • • • , W n , ■ ■ ■} be a sequence of independent identically distributed 
Bernoulli random variables taking on values {0,1} with p = 1/2. Define the map f w as in (1). Then, 
the random variable y = f w (W) has the following properties: 

(i) E[y] =0 

(ii) Var[y] = E[y 2 ] = 1 

(Hi) y has a fractal probability measure which draws values from the interval 

r Vw \fw \ 

[ (l-y/l-w)' (l-x/T^) ;- 

(iv) if w = 3/4, then y is distributed uniformly in {-V3,V3). 
Proof. 

(i) This follows from the fact that the independent Bernoulli trials are fair. 

(ii) Denote a. = \fw] (1 — w) and 8 = y/ (1 — w). Then, w\ = a. ■ B l . Now, observe that E[y 2 ] = 
E~i E Xi [(2Wi - l) 2 ] • w] = a 2 ■ 5Xi j6 2/ = « 2 ■ B 2 /(l - B 2 ) = 1. The contribution to the variance 
due to the variable Xi is w. 

(iii) The interval for y extends symmetrically to a length of YT=\ w i = a ' C^=l ft = a " ]6/(l — fi) — 
sfwl (1 — y/1 — w) about the origin. Now, consider the sequence, W* = {W2, W3, • • • ,W n , ■ ■ ■} 
which can be formed from the sequence W by omitting the first random variable Wi. Clearly, the 
two random variables, y* = /a;(W*) and y = /a;(W) have the same probability density function, 
because W, are iid giving, P[y* /W ) = V(y,w)- But the probability density function P(y /W \ can also 
be expressed in terms of P(y* iW ) m a different way. To see this, observe that W\ = or 1 with 
equal probability, and hence, y = B(y* ± a) with equal chance. From these we get, P{y* tW )(y) = 

P(y,w)(y) = (P{y,w)(y/P ~ a ) + V{y,w)(y/fi + a ))/( 2 i 6 )- This shows that, the pdf of y is formed 
from scaled and translated versions of itself. See Figure 4 for an example with w = 0.05. 

(iv) For the case w = 3/4, ye [-y/3,y/3) from (iii). To prove that y ~ U{[-\/3, a/3)), take W to 
be the terminating binary representation of x = (z + -\/3)/2v / 3 where, z ~ LT([-v/3,\/3)), then by 
construction, y = z for all realizations. 
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Figure 2 The encoded bit stream is mapped to the analog channel code using the function f w (.) at each instant and across all encoder levels. The 
output is a sequence of real valued random variables which unit variance. 
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Figure 3 Block diagram of the Encoder which maps the Analog source output into the code symbols 
in R. Where as the code construction is valid for any countable integer M, due to practical 
considerations, one may have to limit implementations to a finite value of M. 




Figure 4 Histogram showing /?(y /H >=o.05) (y)> with 20000 samples: Self similarity under scaling is apparent. 



We now fix a few notations to be used later. Let us denote by C(p nrr ,j), the Shannon channel capacity 
of a soft output, additive noise channel with a noise pdf of p n (n), at an output SNR of 7, when the 
input alphabet is restricted to be binary. We also denote by P(j /W ), the pdf of y = f w (Y). Define a new 
random variable, z 7fW = /3y + n 7 , where as before ft = yj(l — w). If the random processes y and n 7 
are independent, then the three pdfs are related as, Pz 7/B ,(z) = p n? (^) © V{y,w){y / ft)/ ft, with © denoting 
a linear convolution. 

The bandwidth expansion and the minimum MSE distortion achieved for the analog codes constructed 
as in Section III are related. The scaled nature of the pdf of the noise contributed by the less significant 
bits of the binary representation of j/y leads us to a characterization of the performance of the analog 
codes. 

Theorem IV.l Let a real analog source S draw independently with a uniform distribution from the interval, 
S = [-V3,V3), so that the mean source power is restricted to be unity. Let the source outputs be transmitted 
using a real alphabet, additive noise channel, with a noise pdf of p n (n). Also let the source process be 
independent of the channel noise process. Then for any integer B ^ 1, at a bandwidth expansion factor 
B/C(p z ,wj/(l + (1 — w)j)) it is possible to simultaneously achieve for different integers k ^ 1, MSE 
distortions of (1 — w) k ' B at corresponding SNR of 7/ (1 — w)^ 1 , using capacity achieving channel codes 
designed for an additive noise memoryless channel with noise pdf given by z<y /W . 



Proof. 
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The proof makes use of the explicit code construction presented in Section III. The binary channel 
encoders used in the construction are chosen, so that they achieve the capacity C(p z ,wy/ (1 + (1 — 
w)j)) over the channel with additive noise of pdf p z (z). Recall that, the most significant bit at instant 
j of the transmitted real symbol y,- was denoted by x/qj. The bit yo ; sees a channel which is equivalent 
to one with an additive noise of pdf p Zjw ( z ) an d an output SNR of ivy/ (1 + (1 — w)7), as shown in 
Figure 5. At the receiver, the received value will be fj = Wqi + Zj /W , which can be decoded to form the 
original sequence Uo with an arbitrarily small probability of error, due to the capacity achieving channel 
coding. This in turn recovers the first B bit planes of the source. The MSE distortion is then only due to 
the rest of the bits numbered from B + 1 in the binary representation Xf of the source symbol Sf . This is 
(1 — w) B as claimed. The bandwidth expansion factor is clearly B/C(p z ,wy / (1 + (1 — w)y)). 

MSB Additive Noise Channel 

Var<b) = W v 

, b — <+) ' K + ) ' *~ • 



Var(y*) = (1-w) 

LSBs 



n~N(0,l/y) 



Figure 5 The presence of weighted less significant coded bits has the effect of an added noise of variance 
(1 — w) in addition to the AWGN n 7 ~ A/"(0, 1/7). Here, b is a Bernoulli(p = 0.5) random 
variable taking values from {± v / w}; y* has zero mean, variance (1 — w), and a pdf given by 
V{y,w) (j// ft) / |S- From the figure, the actual SNR for the binary symbol b is, 7' = ivy/ (1 + (1 — 
w )l) ^ 7- 

From above discussion, for an output SNR greater than 7, a distortion of (1 — w) B is achievable for 
a bandwidth expansion factor of N = B/C(p z ,wy/(l + (1 — w)j)). This code recovers without 
error, the binary sequence Ui. From this sequence, we can recover by reordering, the binary sequence 
representation Xy of each symbol of the source sequence, Sj correct up to the first B bits(denoted as S/,b)- 
The perfectly recovered sequence of bits, Ui = {Ui} can be re-encoded at the receiver and subtracted 
from the received sequence, r = {ti = Wy + z 7/K ,}, and then scaled by 1/ B, to obtain a new random 
variable sequence, Y\ = {r^j = YJjj + z '^/(\-w) w } ^ Lemma IV. 1. Invoking the decoder again on t\, 
and since 7/(1 -w) k ^ 7/(1 — w), we can recover W^/ perfectly. We now proceed by induction on the 
number of stages of the decoder operation, denoted by k. 

Let S be a source drawing from an alphabet, S with a probability distribution, ps( s ) which allows 
successive refinement of information [EC91] for a distortion function d, then similar results may also hold 
for such sources. Figure 6 gives a block diagram of the receiver structure proposed in the Theorem IV. 1. 

The D2 versus 7 curve given by the Theorem IV. 1 falls at a rate proportional to y~ B . In the limit as the 
rate of the binary code approaches unity, B — > N, and the distortion falls at a rate proportional to in 
the limit. This means that at high enough SNR, the analog code performance curve can be made parallel 
to the optimal distortion curve for the power limited AWGN channel, given by, D2 ^ 2^e(^ SNR)~ N . 
Therefore the analog code presented in this paper can be expected to outperform some other schemes 
suggested in literature [CW98, VC03] at large 7. 



A. Discussion and Example 

In Figure 7, the capacity curve for a binary input channel with two independent additive noise 
components, one a uniform noise distributed in the interval [— \/3/2, v3/2) and the other a Gaussian 
noise of zero mean and unit variance is shown. In Figure 8 a typical achievable information rate and 
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Figure 6 The decoder for the Analog Code, with M decoding stages corresponding to the M most 
significant bits. In the figure, rt denotes the real valued channel output at time t, while Uj and 
Wj denote the estimates for Uj and Wy (see Figure 3) respectively at the receiver. These estimates 
Uj can be reordered and used to recover an estimate of the analog source output in an obvious 
manner. 

C(bits/use) 



Capacity C 




-10 -5 5 10 (T z 

Figure 7 The capacity of the two additive noise binary input, real output channels. One of the channels 
is the AWGN channel with normal pdf Af(0, cr 2 ) . The second channel has a Gauss-Uniform pdf 
QU{^,cr 2 ). 

corresponding SNR are shown. The noise variance at which a rate of B/p is achieved on this channel is 
denoted by cr 2 . We now restrict our discussion to the simplest case when w = 3/4. In this case, below 
a noise variance of cr 2 , the first level of encoded bit stream can be decoded with arbitrarily small error 
probability when the bandwidth expansion ratio N is at least p. Similarly, the second level of encoded 
bit stream can be decoded with vanishing error probability when the noise variance is below cr 2 /4. In 
general, the i th level can be decoded when the noise variance is below cr 2 /2 l ~ 1 . 

When w = 3/4, the additive noise due to the lower significant bit planes suffered by the bits in the 
first level of encoded bits is distributed uniformly in the interval [—y/3/2, y/3/2). The combined pdf of 
the additive noise presented to the i level of encoded bits is the convolution of the uniform pdf in the 
interval [— with the Gaussian pdf jV(0, cr 2 ), where a\ = In the remaining sections, we will call 
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Figure 8 The capacity of the Gauss-Uniform noise channel with pdf GU(^p, cr 2 ). A typical achievable rate 
point based on the ratio B/p and the corresponding SNR, 1/cr 2 , where N = p is a particular 
bandwidth expansion factor. 

this new pdf the Gauss-Uniform pdf, and denote it by QU(aj,(r 2 ). 

V. Analysis of Finite Length Practical Encoders and Hard Decision ML Component 

Decoders 

We choose the weight factor to be w = |. Consider the case of B = 1 and let all component binary 
codes be the same. According to the analog code construction outlined earlier, the different bit planes of 
the source output are encoded separately using binary codes of rate R, and then the map f w (.) is applied 
to obtain the analog code. The bandwidth expansion factor is then N = 1/R. However the guaranteed 
rate of fall of distortion is only (1 + SNR)^ 1 , because B = 1. 

Now consider the case when B = 2 and assume all component binary codes to be rate R capacity 
achieving binary codes. Bit level reordering, encoding with component codes and mapping with f w (.) is 
shown in the Figure 1 and Figure 2. The rate of fall of distortion is (1 + SNR)~ 2 , because B = 2, while 
the bandwidth expansion factor is N = 2/R. 

Consider the i th level binary encoded sequence X ; . This sequence of coded bits suffers two independent 
additive noise components. One is the uniformly distributed noise due to the lower levels, while the other 
is the AWGN offered by the channel. Let us define a x ■ = Then the coded bits are transmitted at ±flj, 
uniform noise is distributed in the interval [— and the AWGN is Af(0, a 2 ). The probability of an 
encoded bit in the i level being in error can be calculated as follows: 

The combined additive noise pdf due to the uniform pdf and the Gaussian is given by: 




z + Ui 



aV2 



)-erf( 



z — 



aV2 



which is the pdf of a random variable distributed as QU(aj,cr 2 ). 



1 1 



Assuming equally likely encoded ±flj, the probability of an encoded bit in level i being in error is 
given by, 



P(i,cr) = / f z (z)dz = -= lie 2^ dxdt 

lz=o 2ai<j\/2n J- a , 



00 (x+t) 2 



,2 



<r{l-e , 1 . f OiV2 

= -= h -erfc( ) 

2fl fV / 27T 2 v 

We can observe that for large SNR, coded bit error probability is dominated by a, while for small SNR, it 
is dominated by the factor (1 — e^ 2a ^ a ). It is also seen that the probability depends only on the relative 
SNR of the i th encoded bit level, which is a 2 /a 1 . 

Now we consider the case when a systematic binary linear code with parameters [n,k,d] is used as 
the component code. The probability of information bit error under hard-decision ML decoding when the 
code is used over a BSC with crossover probability of p is upper bounded by: 

m (n 
n \m 



m= i=l + l 



Pe^ E i I P^l-Pr^ (2) 



Using the expression for P(i,o~) for the crossover probability p, we get, for the i th level, the information 
bit error is upper bounded as: 



The average distortion due to the i level is given by, 

B-1 B-1 /o 1(/\B _-\\ 

D(z» = 2 £ a\ BH) P e (i,cr) = 2 E(^7y) 2 ^( z » = \ Bl 

;=0 j=0 Z ^ 

the summation above over / was because errors in the i th coded bit level cause errors in the source bits 
planes {(Bi — /) : / e {0, . . . , (B — 1)}}. 

The distortion suffered at each level are independent and add up to the total distortion. If at the receiver, 
the analog decoder is limited to decoding the first I levels, then the corresponding distortion is therefore: 

7 7 

Dj(cr) =2- 2BI + Y^D(i,a) = 4T B1 + 2(4 B - 1) Y^4- Bi P e (i,a) 

i=l i=l 

where the first term is the total distortion due to all source bit planes below level BI. 

The performance of the analog codes constructed here can actually be better than the bound derived 
above if we use soft decision maximum likelihood decoders for the individual component codes. In 
practice, we can replace unbounded length capacity achieving codes with very good algebraic codes or 
with graphical codes. In the second case we have a low complexity near ML decoding algorithm readily 
available in the form of the message passing algorithm. Also, we can truncate the number of source bit 
planes, I which are actually encoded to a manageable, yet reasonably high value. 



VI. Discussion and Examples 



In this section, we examine two practical encoders and their performance to demonstrate the effectiveness 
of the new analog code construction, especially at higher SNRs. 
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A. Perfect Binary Golay Code [23, 12, 7] as Component Code 

The rate of this code is R = 23/12. Since the code is perfect, in the bound of (2), equality holds. 

We consider the case with B = 2. The bandwidth expansion factor, N = jr = f| ~ 3.833. In Figure 9, 
the lower bound on the distortion at a bandwidth expansion factor of N ~ 3.833 and N = B = 2 are 

log w (D 2 ) 




D^p=2) 



\ 

Figure 9 Upper bounds on the mean squared distortion of the proposed analog code constructed using 
the binary Golay code [23, 12, 7] as the component code and hard decision ML decoders for the 
component codes at the receiver, p denotes the bandwidth expansion factor. 

shown, along with the upper-bound on distortion when this particular choice of component code is used 
along with hard decision ML decoding for the component codes at the analog decoder. We can see that, 
even with the number of source bit planes restricted to I = 50, we already see a rate of decline of 
distortion with increasing SNR which is proportional to the N = B = 2 case. 

B. Binary Code [72,36,16] as Component Code 

This code is taken from [MS77]. When B = 3, the bandwidth expansion factor is given by N = B/R = 
6. From Figure 10, we see that the rate of fall in distortion is proportional to the N = B = 3 case. 

For both the codes considered in this section, the actual performance can again be improved if we use 
soft decision decoders for the component codes. 

C. Repeat Accumulate Codes as Component Codes 

As another verification of the code construction and decoding procedure, we employ the fairly powerful 
Generalized Repeat Accumulate (RA m ) code [DJM98], with m = 2 as the binary code to construct an 
analog code. The random permuters used were of length 27000. This code choice enabled the use of a 
factor graph based Sum-Product Algorithm(SPA) [KFL01] at the decoder. But note that any other good 
graph based codes may also be used. Knowing the pdf p z ^ 3/4 (z), the messages from the leaf nodes of 
the Factor Graph (see Figure 11, Figure 13 and Figure 12) can be calculated and the message passing 
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Figure 10 Upper bounds on the mean squared distortion of the proposed analog code constructed using the 
binary code [72, 36, 16] as the component code and hard decision ML decoders for the component 
codes at the receiver, p denotes the bandwidth expansion factor. 



Likelihood from the Channel 




Figure 11 The factor graph for the (RA 2 ) code. 
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Figure 12 The messages into and out of a check (function) node. 




Figure 13 Forward backward algorithm on the Accumulate code graph. Here, p(u) stands for the 
conditional pdf offered by the channel for the output, given an input symbol. Hence, for the 
AnalogCodes, p(u) = p z ^ w \y..(z\y), which is known in closed form for w = 3/4. The variables 
have a binary alphabet. For the truncated Analog Codes, p(u) is a conditional Gaussian pdf . The 
variables in this case are from a 2 M -ary alphabet. 

algorithm now proceeds through a forward backward schedule, which is terminated at a previously fixed 
iteration level. We simulated the sum-product algorithm for 20 iterations to report the curves of Figure 14. 
The simulation results are seen to be in excellent agreement with Theorem IV. 1. The slope of the D2(dB) 
- 7(dB) curve was — B throughout the SNR regime we simulated. The actual simulation curve is worse 
than the predicted performance from Theorem IV. 1, by an SNR(dB) value by which the decoding threshold 
of the (RA 2 ) code under iterative decoding differs from the Shannon limit for that channel. The decoder 
complexity increases linearly as the number of bits of precision, M demanded at the receiver end. In 
a broadcast scenario, the transmitter transmits the same code symbols irrespective of the receiver SNR. 
Since the receiver knows the SNR, it can decode using the SPA at a resolution which can be supported by 
the code at the receiver SNR. Thus, in principle, the receiver can achieve any of the MSE distortion points 
predicted by Theorem IV. 1 . In practice however, the receiver performance is dictated by the performance 
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Figure 14 The code performance for bandwidth expansion N = 4 on AWGN channel SNR. Analog (RA 2 ) 
codes for (B = 2, R = 0.5) and (B = 3, R = 0.75) are plotted along with corresponding lower 
bound for the chaos codes of [CW98], and the Shannon lower bound, D2 ^ ^(1 + SNR)~ N 
for the AWGN channel. The interleaver was of length 27000. 

of the binary component code. In particular, at higher SNRs, error floors of the binary codes can result 
in additional distortion. 

D. Bandwidth efficient communication with truncated Analog Codes 

The effectiveness of analog codes based on capacity achieving component binary codes, prompts one 
to consider truncated versions of these for bandwidth efficient communication. The modulation codes that 
we consider first are analog graphical codes truncated to the M most significant bits. Then for a code rate 
of R, the bits per channel use is seen to be 2M ■ -R for a complex AWGN channel. As a typical example, 
we simulated with generalized (RA 2 ) code as the component codes for the analog code with B = 1 over 
a complex AWGN channel. The channel likelihood messages (see Figure 12, Figure 13 and Figure 11) are 
easily determined, since the channel conditional pdf is known to be Gaussian distributed. At the decoder, 
SPA is again employed (Figure 13). The code alphabet now has 2 M symbols. Hence, each message vector 
is of dimension 2 M . At the end of the stipulated number of iterations, the source symbol with the largest 
associated aposteriori probability is selected. 

A variation of the truncated analog codes has been found to perform better for small values of M (at 
least for M < 4). Here, the Repeat Accumulate codes work on the ring Z 2 m. When M = 1, the resulting 
codes are the same as the binary [RA m ) codes. The mapper converts the symbols in Z 2 m to real numbers, 
using the natural mapping, scaled to achieve the desired average transmit power. The decoder uses the 
SPA( Figure 13). Now the component code trellises for the serial concatenation scheme draw from an 
alphabet with 2 M symbols. Again, for a complex AWGN channel, the channel conditional pdf is known 
to be Gaussian distributed, and at the end of the stipulated number of iterations, the source symbol with 
the largest associated aposteriori probability is selected. The results have been plotted in Figure 15, along 
with the computational cutoff rates of MPSK over complex AWGN channel. 

VII. Comparison with Prior Bounds on Distortion 

In this section, we compare the new codes with the class of signals considered by Ziv. We show that 
the key condition of boundedness of "stretch-factor" does not hold for the codes constructed in this paper. 
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Truncated Analog (RA 2 ) Code Performance and MPSK cutoff rates 
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Truncated Analog (RA 2 ), B=1 ,R=0.5,M=(1,..,4) 




Figure 15 The performance of (a)Truncated Analog (RA 2 ) codes for (B = 1,R = 0.5) and 
M e {1,2,3,4} and (b)Z 2M (RA 2 ) codes for Me {1,2,3,4}, at BER = 1(T 5 are plotted along 
with corresponding computational cutoff rate for MPSK on a complex AWGN channel. The 



interleaver was of length 27000. 



Therefore the threshold effect predicted in [Ziv70] does not necessarily apply to the new codes. 

In [Ziv70] Ziv considers an analog data source which produces a sequence of real samples, S\, Si, ■ ■ ■ 
at a rate of 2W$ samples per second. The encoder maps a block of k = 2TWg such source samples, S, 
using a function, f(t,S) such that the following hold: 

] ^= fmc 



[f(t,s 



< oo 



ijT E s [(f(t,S)) 2 ]dt^l 



If we restrict to the uniform analog source considered earlier, then Theorem 2 in [Ziv70] states that 
given a mapping f(t, S) and positive integers M, ol, Aq such that for every / = l,2,...,k 

I l-T\dfj(t,S,A)\ 2 

' —a J S:Sj 

where 



def 







2a 



-dsidP(S\si)dt ^ MA a 



(3) 



def 



dfj(t, S, A) = f[t, si, s 2/ . . . , Sj + A, . . . , x k ) - f(t, si, s 2 ,...,Sj,..., x k ) 
for any A ^ Aq ^ 2a, then for any sufficiently large SNR 7, and a positive real number K, 

K(2a - A ) 



D > 



(M 7 ) 



-2/a 



The function dj(A) is called the expected stretch factor, and is assumed to be bounded by a number 
proportional to a positive power of A. This assumption is valid for most practical analog communication 
systems such as FM, where there is a bandwidth expansion factor. However, for the analog codes 
constructed in this paper, the stretch factor is not polynomially bounded as a function of A. We will 
show this by means of an example. Assume that all of the component codes are the binary [7,3,4] code, 
which is the dual of the [7,4,3] Hamming code. This code has the generator matrix given by: 

"10 10 10 1" 
10 111 
11110 
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Figure 16 The dual Hamming code [7, 3, 4] is used as a component code. The analog code sequences are 
shown when X\ alone is changing, while Xi = 0.7095 and X3 = 0.4289 




x i^ 

Figure 17 The dual Hamming code [7, 3, 4] is used as a component code. Position number 7 in the analog 
code sequence is shown when x\ alone is changing, while Xi = 0.7095 and X3 = 0.4289 

Figure 16 shows the analog code mapping as a function of X\, while X2 is held fixed at 0.7095 and 
X3 is fixed at 0.4289. Recall that Xj = (sy + v^3)/2v / 3, where Sy are the samples from a band-limited 
analog source with a zero mean, unit variance uniform pdf. A very interesting situation is immediately 
apparent. For example, compare the map when X\ = 0.5 — A and X\ = 0.5. No matter how small the 
A, the entire mapping changes from one point to the other. This is because all the bits in the binary 
representation of 0.5 are different from the bits in the binary representation of 0.5 — A, forcing all the 
component codewords to change. The same situation recurs when X\ approaches all powers of 1/2. In 
fact, the mapping is almost everywhere discontinuous as a function of X\. This is an example of curve 
which is self-similar under scaling and shifting. Such forms are well known as fractals [Man82]. Similar 
observations can be found valid on all of the other three source dimensions as well. 

Therefore, it is clear that the analog codes constructed in this paper do not fall in the class of modulation 
signals considered in [Ziv70]. 
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One can also observe that, the proposed analog codes do not touch the corresponding rate-distortion 
based lower bounds on MSE distortion. Therefore, the conclusions similar to those of Reznic et al of 
[RFZ05] do not apply as well. Moreover, in [RFZ05], they consider Gaussian sources, whereas our 
construction is for a uniform source, though it may be possible to extend it to any source which admits 
successive refinement of information [EC91]. 

VIII. Summary 

We presented a new analog coding scheme, which can achieve a mean-squared error distortion 
proportional to (1 + SNR)~ B for a bandwidth expansion factor of B/R, where < R < 1 is the 
rate of individual component binary codes used in the construction. Thus over a wide range of SNR 
values, the newly proposed code performs much better than any single previously known analog coding 
system. 
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